Skip to content

fix(core): split PRP chunks by DMA address, not user page offset - #23

Closed
sungjoo-XCENA wants to merge 1 commit into
mainfrom
sungjoo/fix/prp_dma_offset
Closed

fix(core): split PRP chunks by DMA address, not user page offset#23
sungjoo-XCENA wants to merge 1 commit into
mainfrom
sungjoo/fix/prp_dma_offset

Conversation

@sungjoo-XCENA

Copy link
Copy Markdown

Problem

mx_prp_first_chunk_len() computed the first PRP chunk length from the CPU-side page offset (sg->offset), but the chunk itself is placed at sg_dma_address(). The device derives the first chunk length from the DMA address it receives, so whenever the mapping does not preserve the buffer's low address bits, every PRP entry after the first points at the wrong address. Any transfer needing more than one PRP entry then silently corrupts data — shifted by the alignment delta — while the ioctl still returns 0.

The mismatch occurs when user_offset % dma_size != dma_addr % dma_size. Direct mapping and IOMMU translation preserve the intra-page offset, so the bug stays hidden there; SWIOTLB bounce buffering does not (no dma_min_align_mask is set), so it corrupts. SWIOTLB engages with no IOMMU + buffers outside the device's reach, in confidential VMs (SEV/TDX), or with swiotlb=force.

Reproduction (QEMU guest, no vIOMMU, SWIOTLB active)

2576-byte DeviceInfo read on the v1 queue (dma_size=1024), same run, only buffer offset differs:

user buffer sg_dma (bounced) first_len desc list emitted
page_offset=0 0x7cfe9000 (aligned) 1024 correct
page_offset=1 0x7e073800 (slot-aligned) 1023 0x7e073800, 0x7e073bff, 0x7e073fff

The device writes 1024 bytes from 0x7e073800 (address is 1 KiB-aligned), then jumps to the second entry 0x7e073bff, overwriting the byte it just wrote: all data past byte 1023 shifts down by one (hmboxRqOffset reads back 0x10 instead of 0x1000; payload FNV-1a hash differs from the aligned run). ioctl returns 0 in both cases. A read-size matrix confirms the boundary: sizes ≤ 1024 pass at any offset, sizes ≥ 1025 corrupt at offsets 1/511/1023.

Fix

  • mx_prp_first_chunk_len(): derive the split from sg_dma_address(sg) + intra_off instead of sg->offset. This fixes all callers (v1 single/multi decision, v2 two-entry path, desc-list walk and count).
  • create_mx_command_sg() (v2): branch on the DMA-side descriptor count (mx_get_total_desc_count()) instead of a host page count computed from the user virtual address — same failure mode. The SINGLE_DMA_SIZE == PAGE_SIZE static_assert only guarded that host-page branching and goes away with it.
  • transfer.c still uses the user offset for page pinning, which is CPU-side and correct.

With the fix the same transfer emits 0x7e073800, 0x7e073c00, 0x7e074000.

Testing

  • Clean module build against 6.8.0-124-generic.
  • Arithmetic cross-check: a userspace transcription of the descriptor-walk logic reproduces the exact buggy desc list from the QEMU dmesg above; with this change the list is boundary-aligned and a modeled PRP consumer reconstructs all 2576 bytes intact.
  • Real-HW note: hosts mapping the device through an IOMMU DMA domain (checked on gnr-pxl, DMA-FQ) are unaffected before and after — behavior only changes where SWIOTLB previously corrupted.

mx_prp_first_chunk_len() derived the first chunk length from the
CPU-side page offset (sg->offset), while the chunk itself is placed at
sg_dma_address().  The device computes the first chunk length from the
DMA address it receives, so whenever the mapping does not preserve the
buffer's low address bits -- e.g. SWIOTLB bounce buffering -- every PRP
entry after the first pointed at the wrong address.  Transfers needing
more than one PRP entry then silently corrupted data (shifted by the
alignment delta) while the ioctl still returned success.

Reproduced in a QEMU guest (no vIOMMU, SWIOTLB active): a 2576-byte
DeviceInfo read at user page offset 1 was bounced to a 1 KiB-aligned
DMA address; the driver emitted the desc list [base, base+1023,
base+2047] instead of [base, base+1024, base+2048], shifting all data
past byte 1023 by one.  Direct mapping and IOMMU setups preserve the
intra-page offset, which is why the bug stayed hidden on real hardware.

Also branch create_mx_command_sg() (v2) on the DMA-side descriptor
count instead of a host page count computed from the user virtual
address, which could disagree with the DMA layout the same way.  The
SINGLE_DMA_SIZE == PAGE_SIZE static_assert only guarded that host-page
branching and goes away with it.
@sungjoo-XCENA
sungjoo-XCENA deleted the sungjoo/fix/prp_dma_offset branch August 12, 2026 08:45
@sungjoo-XCENA

Copy link
Copy Markdown
Author

Wrong repo — resubmitted to the dev repo: https://github.com/xcena-dev/mxdriver_dev/pull/86

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant